Papers with multi-objective alignment
Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Prompts (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for aligning large language models with human preferences are poor in extensibility and require significant retraining. |
| Approach: | They propose a multi-objective alignment approach that constructs an expert prompt and an adversarial prompt for each alignment objective to contrast at the decoding time. |
| Outcome: | The proposed approach is superior to existing methods in obtaining a well-distributed Pareto front among different alignment objectives. |
Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment (2024.emnlp-main)
Copied to clipboard
Yiju Guo, Ganqu Cui, Lifan Yuan, Ning Ding, Zexu Sun, Bowen Sun, Huimin Chen, Ruobing Xie, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing algorithms for achieving optimal alignment are mostly unidirectional . a recent study suggests that large language models can be ground with evident preferences . |
| Approach: | They propose to ground large language models with evident preferences . they propose to use controllable preference optimization to specify different objectives . |
| Outcome: | The proposed models can provide responses that match various preferences among the ”3H” desiderata. |
APEX: Learning Adaptive Priorities for Multi-Objective Alignment in Vision-Language Generation (2026.findings-acl)
Copied to clipboard
Dongliang Chen, Xinlin Zhuang, Junjie Xu, Luojian Xie, Zehui Wang, Jiaxi Zhuang, Haolin Yang, Liang Dou, Xiao He, Xingjiao Wu, Ying Qian
| Challenge: | APEX optimizes for text-to-image generation by combining learning potential, conflict penalty, and progress need. |
| Approach: | They propose an algorithm that stabilizes heterogeneous rewards and dynamically schedules objectives . they propose a method that achieves better Pareto trade-offs across four heterogenous objectives based on P3 Adaptive Priorities . |
| Outcome: | The proposed algorithm achieves better pareto trade-offs across four heterogeneous objectives while maintaining competitive OCR accuracy. |